Non-Asymptotic Optimal Guaranteed MSE Function Estimation

Local models and methods construct function estimates or predictions from observations in a local neighborhood of the point of interest. The bandwidth, i.e., how large the local neighborhood should be, is often determined based on asymptotic analysis. In this report, an alternative, non-asymptotic approach that minimizes a uniform upper bound on the mean square error for a linear or affine estimate is proposed. Three different classes of problems are considered, based on what degree of a priori information is given, and the relations between them are studied. It is shown that the solution is obtained from a quadratic program or, for some classes of problems, from a second-order cone program, and that it maintains many of the key features of the asymptotic approaches. Furthermore, it is shown that the approach has a desirable property of automatic selection of bandwidth, which could also be used to enhance the computational efficiency. Moreover, examples show that the proposed approach in some cases is superior to an asymptotically based local linear estimator. Finally, the problem of estimating function derivatives using the same approach is considered.